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Abstract 

The literature on sparse recovery often adopts the £ p “norm” (p £ [0,1]) as the 
penalty to induce sparsity of the signal satisfying an underdetermined linear system. 
The performance of the corresponding E p minimization problem can be characterized 
by its null space constant. In spite of the NP-hardness of computing the constant, 
its properties can still help in illustrating the performance of i v minimization. In this 
letter, we show the strict increase of the null space constant in the sparsity level k 
and its continuity in the exponent p. We also indicate that the constant is strictly 
increasing in p with probability 1 when the sensing matrix A is randomly generated. 
Finally, we show how these properties can help in demonstrating the performance of £ p 
minimization, mainly in the relationship between the the exponent p and the sparsity 
level k. 

Keywords: Sparse recovery, null space constant, £ p minimization, monotonicity, 
continuity. 


1 Introduction 

An important problem that often arises in signal processing, machine learning, and statistics 
is sparse recovery W- It is in general formulated in the standard form 

argmin||x||o subject to Ax = y (1) 

X 

where the sensing matrix A £ M MxAr has more columns than rows and the £q “norm” ||x||o 
denotes the number of nonzero entries of the vector x. The combinatorial optimization 0 
is NP-hard and therefore cannot be solved efficiently [4j. A standard method to solve this 
problem is by relaxing the non-convex discontinuous £q “norm” to the convex i\ norm [5], 
i.e., 

argmin||x||i subject to Ax = y. (2) 

X 
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It is theoretically proved that under some certain conditions 15,6], the optimum solution of 
([2]) is identical to that of (jTj). 

Some works try to bridge the gap between £0 “norm” and £\ norm by non-convex but 
continuous £ p “norm” (0 < p < 1) [7j-[l0|, and consider the £ p minimization problem 


argmin||x||jj subject to Ax = y 


(3) 


where ||x|| p = X^i=i \ x i\ p ■ Though finding the global optimal solution of £ p minimization is 
still NP-hard, computing a local minimizer can be done in polynomial time [II]. The global 
optimality of ([3]) has been studied and various conditions have been derived, for example, 
those based on restricted isometry property BUI and null space property [To], 13]. Among 
them, a necessary and sufficient condition is based on the null space property and its 
constant |10l|13l|14|. 


Definition 1. For any 0 < p < 1, define null space constant 7 (£ p ,A,t’) as the smallest 
quantity such that 

£>| p <7(4»A ,k)^2\ Zi \P (4) 

i€S i£S 

holds for any set S C {1,2,..., A} with #S < k and for any vector z £ AA(A) which 
denotes the null space of A. 


It has been shown that for any p £ [0,1], j(£ p ,A,k) < 1 is a necessary and sufficient 
condition such that for any fc-sparse x* and y = Ax*, x* is the unique solution of £ p 
Therefore, 7 (£ p , A, k) is a tight quantity in indicating the performance 


minimization 
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of £ p minimization (0 < p < 1) in sparse recovery. However, it has been shown that 
calculating 7 (T p , A, k ) is in general NP-hard 15 , which makes it difficult to check whether 


the condition is satisfied or violated. Despite this, properties of 7 (T P , A, k) are of tremendous 
help in illustrating the performance of £ p minimization, e.g., non-decrease of ^{£ p , A, k ) in 
p £ [0,1] shows that if £ p minimization guarantees successful recovery of all k -sparse signal 
and 0 < q < p, then £ q minimization also does [lOj. 

In this letter, we give some new properties of the null space constant y(T p , A, k). Specifi¬ 
cally, we prove that 7 <T p , A, k ) is strictly increasing in k and is continuous in p. For random 
sensing matrix A, the non-decrease of j(£ p ,A, k) in p can be improved to strict increase 
with probability 1. Based on them, the performance of £ p minimization can be intuitively 
demonstrated and understood. 


2 Main Contribution 

This section introduces some properties of null space constant 'y(£ p ,A,k) (0 < p < 1). 
We begin with a lemma about 'j(£ p ,A, k) which will play a central role in the theoretical 
analysis. The spark of a matrix A, denoted as Spark(A) [l6|, is the smallest number of 
columns from A that are linearly dependent. 
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Lemma 1. Suppose Spark(A) = L + 1. For p E [0,1], 

1) 7A, fc) is finite if and only if k < L; 

2) For k < L, there exist S' C {1,2,..., N} with #S' < k and z! E AA(A) \ {0} such 
that 

ieS" igS' 


Proof. See Section 3.1 


□ 


First, we show the strict increase of 7 (£ p , A, k) in k. 


Theorem 1. Suppose Spark(A) = L+l. Then for p E [0,1], 7 (£ p , A, k) is strictly increasing 
in k when k < L. 


Proof. See Section 3.2 


□ 


Remark 1. For any p E [0,1], we can define a set fC p ( A) of all positive integers k that every 
k- sparse x* can be recovered as the unique solution of £ p minimization ^ with y = Ax*. 
According to TheoremjlJ IC P ( A) contains successive integers starting from 1 to some integer 
k*( A) and is possibly empty. 

Remark 2. If Spark(A) = L + 1, then k^{ A) = [L/ 2j 16]. Therefore, if L > 2, fcg(A) > 1. 

Remark 3. For A with identical column norms, if Spark(A) = L + 1 and L > 2, then 
*i(A) > 1 . To show this, we only need to prove that 7 (^ 1 , A, 1) < 1. First, for any 
1 < i < N and z E AA(A) \ {0}, since Az = 0, Zi a* = — ’Ylj^i z j aL j where a* is the ith 
column of A. Since 


| 1 12 — | pi a i||2 — 






2 
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with equality holds only when aj (j i) are all on the same ray, which cannot be true 
since Spark(A) = L + 1 > 3. Since A has identical column norms, |zj| < \ z j\ holds, 

which leads to 7 (^ 1 , A, 1) < 1 because of Lemma [l]2) . 

Now we turn to the properties of 7 (^ p , A, k ) as a function of p. The following result 
reveals the continuity of 7 (£ p , A, k ) in p. 


Theorem 2. Suppose Spark(A) = L + 1. Then for k < L, 7 ^, A ,k) is a continuous 
function in p E [0,1]. 


Proof. See Section 3.3 


□ 


Remark 4. Some works have discussed the equivalence of £q and £ p minimizations. In 
| IT], it is shown that the sufficient condition for the equivalence of these two minimization 
problems approaches the necessary and sufficient condition for the uniqueness of solutions of 
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Figure 1: The figure shows k*(A.) as a function of p, where the argument A is omitted for 


concision. 


Co minimization. In |7], it is shown that for any /c-sparse x* and y = Ax*, if 52 fc+i < 1) then 
there is p > 0 such that x* is the unique solution of C p minimization. This result is improved 
to 82 k < 1 which is optimal since it is exactly the necessary and sufficient condition for x* 
being the unique solution of Co minimization Jl2| . [18] shows the equivalence of the Co- and 
the £ p -norm minimization problem for sufficiently small p. According to Theorem [2j we can 
also justify this result: For any fc-sparse x* and y = Ax*, if 7(^01 A, k ) < 1, then there is 
p > 0 such that 7 ^, A, k) < 1 and x* is the unique solution of C p minimization. 

Remark 5. In 1101, the author defines a set 7\(A) of reconstruction exponents, that is the set 
of all exponents 0 < p < 1 for which every fc-sparse x* is recovered as the unique solution 
of C p minimization with y = Ax*. It is shown that Vk{ A) is a (possibly empty) open 
interval (0,p|.(A)) 1101. This result can be easily shown by Theorem[2j Since ^(Cp, A,/c) is 
a non-decreasing [13] continuous function in p G [0,1], the inverse image of the open interval 
(—00, 1) is also an open interval of [0,1]. Therefore, the requirement that ^{C p , A, k) < 1 is 
equivalent to p e [0,p£(A)). 

Remark 6 . For any A, we can plot k*( A) as a function of p, as shown in Fig.JTJ For concision, 
we omit the argument A in the figure. It is obvious that k*( A) is a step function decreasing 
from fcg(A) to A:*(A). Three facts needs to be pointed out. First, k*( A) is right-continuous, 
which is an easy consequence of Theorem [ 2 ] Second, the points (po,ko) corresponding to 
the hollow circles in Fig. [I] satisfy 7 (A po , A, ko) = 1. Third, for the p-axis po of the points 
of discontinuity, the one-sided limits satisfy lim p )p ~ k*( A) — lim p ^ p + k*( A) = 1. This can 
be proved by Theorem [l] that if 7 (^ Po , A, ko) = 1, then 7 (£ po , A, ko — 1) < 1. 

Finally, we introduce an important property of 7 (£ p , A, k) as a function of p with regard 
to random matrix A. 
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Figure 2: This figure shows a diagrammatic sketch of 7 ^, A, k) as a function of p for 
different k when A is a random matrix. 


Theorem 3. Suppose the entries of A G M. MxN are i.i.d. and satisfy a continuous prob¬ 
ability distribution. Then for k < M, j(£ p , A.,k) is strictly increasing in p E [0,1] with 
probability one. 


Proof. See Section 3.4 


□ 


Remark 7. It needs to be noted that there exists A such that 7 ^, A, A;) is a constant 
number for all p E [0,1]. For example, for 


A = 


1 

V2 


1 

1 


( 6 ) 


Spark(A) = 2. Since A7(A) = span([l, —1] T ), it is easy to check that for all p E [0,1], 
7 ( 4 , A, 1) = 1. 

Remark 8. To sum up, we can schematically show 7 (£ p , A, k ) as a function of p for different 
k in Fig. [2] According to Theorem[l] these curves are strictly in order without intersections. 
Theorem [ 2 ] reveals that 7 (£ p ,A,fc) is continuous in p. For a random matrix A with i.i.d. 
entries satisfying a continuous probability distribution, 7 ^, A,k) is strictly increasing in 
p with probability 1 by Theorem [3j According to the definition of k*( A), the curves inter¬ 
secting 7 (T p , A, k) = 1 (0 < p < 1) are those with Afj'(A) + 1 < k < fcg(A). According to 
the definition of p* k ( A), the p-axis of these intersections are p * k *, V*k*_\i • • ■, P*k*+\ from left 
to right. Therefore, it is easy to derive Fig. [l] based on Fig. [2] when A is a random matrix. 
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3 Proofs 


3.1 Proof of Lemma |T| 


Proof. 1) Since Spark(A) = L + 1, JV’(A) contains an (L + l)-sparse signal, and it is easy 
to show that for any k > L + 1, j(£ p , A, k) = +oo according to Definition [lj Next we prove 
that for k < L, j(£ p , A, k) is finite. Define 


6(p,z,S) 


\ z i\ p 


(7) 


and Afi (A) = A7(A) n {z : ||z ||2 = 1} which is a compact set. Then it is easy to see that 
the definition of null space constant is equivalent to 


j(£ p ,A,k) = max sup 9(p,z,S). ( 8 ) 

# S < k z£jVi{ A) 

If 7 (^ p , A, k) is not finite, then there exists S' with ffS' < k such that sup zg _^/- 1 ( A ) 9{p, z, 5') 
is not finite. Therefore, for any n E N + , there exists z^ E A/i(A) such that 

9(p,z^,S')>n. (9) 


If p = 0, since z ^ is at least (L + l)-sparse, it is easy to see that 9(0,z ( ' n ' ) , S') < k holds 
for any n E N + . This contradicts (|9]) when n > k. If p 6 (0,1], according to Lemma 4.5 
z ( n ) I 


m 


10 


ip — 


<Np 2 ||z^ n ^ 11 p = Np 2 . 


and © implies 


E 

i£S' 


Mr - Nl ~ l 


- \ p < 
l I — 


n + 1 


( 10 ) 


Due to the compactness of N\ (A), the sequence {z ^ n '} n has a convergent subsequence 
{z( nm )} m , and its limit zl also lies in A/i(A). Then (10) implies z[ = 0 for * 0 S', i.e., Ai(A) 
contains a k- sparse element z!. This contradicts the assumption that Spark(A) = L + l > k. 
2) If p = 0, for any S with ffS < k and any z E jV(A) \ {0}, it holds that 


0(0, z, S ) < 


k 


L + 1 — k 


( 11 ) 


On the other hand, since Spark(A) =11 + 1, A/"(A) contains an (L + l)-sparse signal zl 
with T as its support set. For any S' C T with ffS' = k, 9(0,z',S') = k/(L + 1 — k ), and 
therefore ([5]) holds. 

If p E (0,1], recalling the equivalent definition ([ 8 ]), there exists S' with ffS' < k such 
that 

'y(£ p ,A,k)= sup 0(p,z,S'). ( 12 ) 

zeA/i(A) 

Since A/i(A) is compact and the function 9(p, z, S') is continuous in z on A/i(A), it is easy 
to show that there exists z! E A/i(A) such that 7 (l v , A, k) = 6{p, z !, S'). □ 
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3.2 Proof of Theorem [Q 

Proof. We prove that when p G [0,1] and 2 < k < L, 


7 ( 4 > A,k - 1) < 7(4, A,k). (13) 

According to Lemma [l]2) , there exist S' with fj^S' < k — 1 and zl G A/i(A) such that 

7(4, A,k - 1) = 0(p,z',S'). (14) 

Since z! is at least (L + l)-sparse, there exists an index s' G {1,2,... ,N} \ S' such that 
z' s , 0. Let S" = S' U {s 7 }, then 


and hence 


E w < E W’ E M” > E i 2 :i p > 0 

«eS" i£S" i<£S' i£S" 


9(p,z',S') < 9(p,z',S"). 


(15) 


(16) 


Recalling (14) and the equivalent definition we can get (13) and complete the proof. □ 


3.3 Proof of Theorem [2] 

Proof. According to Theorem 5 in 1131, 7(4) A, k) is non-decreasing in p G [0,1] and there¬ 
fore can only have jump discontinuities. We show this is impossible by two steps. 

First, for any p G (0,1], we prove the one-sided limit from the negative direction satisfies 

L~ := lim 7(4, A,fc) =7(4, A,fc). (17) 

q^p~ 

According to Lemma [lj2) , there exist S' with fj^S' < k and zl G A/i(A) satisfying 

7 ( 4 , A, k) = 6(p, z ', S'). (18) 

According to the definition of 9{p, z, S), it is easy to show that 

lim 9(q,z', S') = 9{p,z , S'), (19) 

q-^p~ 

and then © holds obviously. 

Second, for any p G [0,1), we prove the one-sided limit from the positive direction 
satisfies 

L + \= lim 7 ( 4 , A, k) = 7 ( 4 , A, k). (20) 

Since p < 1, there exists No G N + such that p + N^ 1 < 1. Then for n > Nq, Lemma [ 1 J 2 ) 
reveals that there exist S^ with ffS^ < k and z^ G A/i(A) such that 

7(W 1 > = 9 (P + n~\z^,S^). (21) 
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Since there are only finite different S satisfying #S < k, there exists S' with #S' < k such 
that an infinite subsequence of {z O0} n is associated with S'. Due to the compactness of 
A/i(A), this subsequence has a convergent subsequence {zand its limit z! also lies 
in A/) (A). According to the definition of 9(p,z,S) and plj ), 

9(p,z',S')= lim 9{p + n” 1 , S') = L + , (22) 


and consequently j(£ p , A, k) > L + . Since 7 (f p , A, k ) is non-decreasing in p, 7 (£ p , A, &) < L + 
and ( 20 ) is proved. □ 


3.4 Proof of Theorem |3] 

Proof. First, we show that Spark(A) = M+ 1 with probability 1. Let M.(M) denote the M 2 - 
dimensional vector space of M x M real matrices. For any 0 < k < M, let Affc(M) denote 
the subset of A4(M) consisting of matrices of rank k. It can be proved that Affc(M) is an 
embedded submanifold of dimension k(2M — k ) in Jvi(M) [l9|. Consequently, for M x M 
matrices with i.i.d. entries drawn from a continuous distribution, the M 2 -dimensional 
volume of the set of singular matrices A4fc(M) is zero. In other words, any M, or 

fewer, random vectors in M. m with i.i.d. entries drawn from a continuous distribution are 
linearly independent with probability 1. On the other hand, more than M vectors in M. m 
are always linearly dependent. Therefore, Spark(A) = M + 1 with probability 1. 

Next, with the equivalent definition ([8]), we prove that for k < M and 0 < p < q < 1, 

max sup 9(p,z,S)< max sup 9(q,z,S) (23) 

#S<k zeM(A ) #s<k zeM(A) 

holds with probability 1. According to Lemma [T]2) , there exist S' with #S' < k and 
zl 6 A/i (A) such that 

9(p, z!, S') = max sup 9(p,z,S). (24) 

# s < fe zeM(A) 

Suppose zl has N* nonzero entries with T as its support set, then A* > M + 1 with 
probability 1. It is obvious that S' C T, and for any i G S' and any l £ T\S' , |z(| > |z(| > 0. 
Since p < q, \z' i \ q ~ p > \z' l \ q ~ p and therefore 

1 4\ q \z'i\ p > \zW\z'i\ q . (25) 


Summing (25) with i in S' and l in T \ S', we can obtain 


E141’ E a E W E I4I" 

ieS' leT\S' i£S' leT\S' 


(26) 


which is equivalent to 


9(p,z',S') < 9(q,z',S'). 


( 27 ) 




Since p < q, it is easy to check that the equality in (27) holds only when \z[\ = \z[\ for all 
i G S' and all? G T \ S', i.e., the nonzero entries of zl have the same magnitude. We prove 
that A/i(A) contains such zl with probability 0, which together with (24) imply that 


7 ( 4 , A, k) = 6(p, z', S') < 0(q, z' , S') < 7 ( 4 , A, k ) 


(28) 


holds with probability 1 . 

To this end, let A4(M, N) denote the MA-dimensional vector space of M x N real 
matrices. For fixed z G R^ with ||z ||2 = 1, it can be easily shown that the subset 


M Z (M, N) = {A G M(M,N) : Az = 0} 


(29) 


is an M(N — l)-dimensional subspace in A4(AI. N ). Therefore, for A G A4(M, N ) with i.i.d. 
entries drawn from a continuous probability distribution, A/i(A) contains z with probability 
0. In {z G : 11z ||2 = 1}, the number of vectors whose nonzero entries have the same 
magnitude is 



(30) 


which is a finite number. Therefore, with probability 0, A/"i(A) contains a vector zl which 
makes the equality in (27) hold. That is, y(^ p , A, k) is strictly increasing in p G [0,1] with 
probability 1 . □ 


4 Conclusion 

In characterizing the performance of ( p minimization in sparse recovery, null space constant 
7 (£ p , A, k) can be served as a necessary and sufficient condition for the perfect recovery 
of all /c-sparse signals. This letter derives some basic properties of j(£ p , A, k) in k and 
p. In particular, we show that j(£ p , A, k) is strictly increasing in k and is continuous in p, 
meanwhile for random A, the constant is strictly increasing in p with probability 1. Possible 
future works include the properties of ^f(£ p , A, k) in A, for example, the requirement of 
number of measurements M to guarantee j(£ p ,A,k) < 1 with high probability when A is 
randomly generated. 
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